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PHFR020136 EPp 

"VIDEO ENCODING METHOD" 

RELD OF THE INVENTION 

The present Invention generally relates to the field of data compression 
and, more spedfically, to a method of encoding a sequence of frames, composed of 
picture elements (pbcels), by means of a three-dimensional (3D) subband decomposition 
involving a filtering step applied, in the sequence considered as a 3D volume, to the 
spatial-temporal data which correspond in said sequence to each one of successive 
groups of frames (GOFs), these GOI^ being themselves subdivided Into successive pairs 
of frames (POft) Including a so-called previous frame and a so-callerf current frame, 
said decomposition being applied to said GOEs together with motion esBmatfoh and 
compensation steps performed in each GOF on salds POFs and on corresponding pairs 
of low-frequency temporal subbands (POSs) obtained at each temporal decomposition 
level. 

The Invention also relates to a computer programme comprising a set of 
instructions for the Implementation of said encoding metfiod, when said programme is 
earned out by a processor included In an encoding device. 

BACKGROUND OF THE INVENTION 

In recent years, three-dimensional (3D) subband analysis, based on a 3D, 
or (2D+t), wavelet decomposition of a sequence of frames considered as a 3D volum 
has been more and more studied for video compression. The coefficients generated by 
the wavelet transform constitute a hierarchical pyramid in which the spatio-temporal 
relationship is defined thanks to 3D orientation trees evidencing the parent-offspring 
dependendes between coefildents, and the in-depth scanning of tiie generated 
coefficients In tfie hierarchical trees and a progressive bitplane encoding technique lead 
to a desired quality scalability. The practical stage for tills approach fe to generate 
motion compensated temporal subbands using a simple two taps wavelet filter, as 
illustrated in Rg.l for a group of frames (GOF) of eight frames. 

In the Illustrated implementation, tiie input video sequence is divided into 
Groups of Frames (GOFs), and each GOF, itself subdivided into successive couples of 
frames (tiiat are as many Inputs for a so-called Motion-Compensated Temporal Filtering, 
or MCTF module). Is first motion-compensated (MC) and Uien temporally filtered (7F). 
The resulting low frequency (L) temporal subbands of the firet temporal decomposition 
level are further filtered (TF), and the process may stop after an arbitrary number of 
decompositions resulting in one or more low frequency subbands called root temporal 
subbands (in the illustration, an example witii two decomposition levels resulting In two 
root subbands is presented). In the example of Rg.l, the frames of the illustrated group 
are referenced Fl to F8, and ttie dotted anrows correspond to a high-pass temporal 
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filtering, while the other ones correspo nd to a low-pass tempera! filtering. Two stages of 



decomp»5.uon are snown (l and H = first stage ; LL and m = second stage). At each 
temporal decoitipositfon level of the illustrated group of 8 frames, a group of motion 
vector fields is generated (In the present example, MV4 at the first level, MV3 at the 
second one). 

.When a Haar mulfaresolutton anateb te used for the temp »r;.i 



decomposition, since one mobon vector field Is generalBd between every two fiames in 
the considered group of frames at each temporal decomposition level, the number of 
motion vector fields is equal to half the number of frames In the temporal subband, I e 
fouratthefirstlevelofmotfonvectorfieldsandtwoatthesecondone. Motion 
• estimation (M^ and motion compensation (I^Q are only perfbmied every two Irames 
of the input sequence (generally In the ibrward way),.due to the temporal dbwrt^ . 
sampling by two of the simple wavelet filter. Using these very simple filter?, each low. 
frequency temporal subband (L) represents a temporal average of the Input couples of 
fiames, whei«as the high frequency one (H) contains the residual error after the MCTF 
-step. 

Unlbrtunately, the motion compensated temporal filtering may raise the 
problem of unconnected picture elements (or pixels), whfch are not filtered at all (or 
also the problem of double-connected pixels, which are filtered twice). TTie number of 
unconnected pixels represents a weakness of a 3D subband codec approaches because 
it highly impacts the resulting picture quality (particularly In occlusion regions). It is 
espedaliy true for high motion sequences or for final temporal decomposition levels 
where the temporal correlation Is.not good. The number of these unconnected pbceb 

depends on the dense motion vector field that has been generated by the motion 
estimation. 

Qiirent criteria for optimal motion vector search used in motion estimators 
do not take into account the number of unconnected pixels that will be the result of 
motion compensation. Most sophisticated algorithms use a rate/distortion criterion 
which tends to minimize a cost function that depends on the displaced diflerence 
energy (distortion) and the number of bits spent to transmit the motion vector (rate). 
For example, the motion search returns the motion vector that minimtees : 



(1) 



Wfth m = (m„«,/ being the motion vector, p = (p^,p^f being the prediction for the 
motion vector, and b^ng the Lagrange multiplier. TTie rate term i?(in-p) 
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represents the motion information only and SAD is used as distortion measure. It is 
computed as : 

SAD(s,c{m))= J^\s[x,y]-c[x'-m^,y^mJ\ (2) 

With s being the original video signal, c being the coded video signal and B being the 
block size (note that B can be 1). Unfortunately, these algorithms do not take Into 
account the distortion introduced by unconnected pixels during the inverse motion 
• compensation because usually these optimizations are applied to hybrid coding for which 
the inverse motion compensation is not perfomied. * 

SUMMARY OF THE INVENTION - 

It is therefore an object of the invention to avoid such a drawback and to 
propose a video encoding method in which the set of unconnected pixels is taken into 
account in the distortion measure. 

To this end, the inventfon relates to a method of encoding a sequence of 
frames, composed of picture elements (pixels), by means of a three-dimensional (3D) 
subband decomposition Involving a filtering step applied, in the sequence considered as a 
3D volume, to the spatial-temporal data which correspond in said sequence to each one of 
successive groups of frames (GOFs), these GOFs being themselves subdivided into 
successive pairs of frames (POFs) including a so-called previous frame and a so-called 
cunient frame, said decomposition being applied to said 60ft together with motion 
estimation and compensation ste^^ perfonmed In each GOF on sakls POft and on 
corresponding pairs of low-frequency temporal subbands (POSs) obtained at each temporal 
decomposition level, this proc^ of motion compensated temporal filtering leading In the 
prevtous frames on the one hand to connected pixels, that are filtered along a motion 
trajectory con^ponding to motion vectors defined by means of said motion estimation 
steps, and on the other hand to a residual number of so-called unconnected pixels, that are 
not filtered at all, each motion estimation step comprising a motion search provided for 
retuming a motion vector that minimizes a cost foncHon depending at least on a distorslon 
aiterion involving a distortion measure, said measure distora'on befng also applied to tiie 
set of said unconnected pixels. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will now be described, by way of example, With reference to tiie 
acoampanylng drawing in which : 

- Rg.l shows a temporal multiresolution analysis wltti moti'on compensation. 
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DETAILED DESCRIPHON OF THE INVENTION 

Because unconnected pixels highly parbapate to the quality degradation of 
the inverse motion compensated image, the set of unconnected pixels is, according to the 

^"v«"t.on, taken into account in the dist ortion measure. To this end, it is here prorx>sed to 

Introduce a new rate/distortion criteHon that extends equation taking Into account.the 

unconnected pixels phenomenon. This is illustrated In equations (3) and (4) : 

/C(m) = 7(in) + XuHcama^ ■ oiSuKomiEaw (m)) «) 

"^f* -^^CWjMfficwaCm)) being the dlstortton measure for the set Si,i^,)i„^ of- 
unconnected pixels resulting from mptlon vector m . Several distprtion measures can be 
applied to the set of unconnected pixels. A very simple measure is preforabV the count of 
unconnected pixels for the motion vector under study. 

It can be noted that the real set of unconnected pixels resulting from a motion 
search can be computed on V when the motion vectors information is available for the 
whole liame. Therefore, an optimal solution can hardV be achievable (in fact a complex set 
of minimisation criteria for the whole frame should be iiolved), and a su^opOmal . 
implementation is therefore piDposed. 

This implementation is not recursive and can be considered as a simple way to 
take Into aoaunt the distortion due to unconnected pixels. For a given partof the Image Id 
be motibn compensated (a part of the image can be a pixel, a btock of pixels , a 

maooblockofplxelsoranyregionprxMdedthatthesetofpartscoveisthewtolefma^^ 
without any overlapping) and for a given motion vectx>r candidate m , a temporary imrerse 
motion compensation is applied, the set of unconnected pixels is identified and 
^(SmcomEciED (m)) can be evaluated. The current K(m) value can be computed and 
compared to 0,e current minimum value K^(m) to check if tiie candMate motion vector 
brings a lower K(m) value. When all the candidate have been tested, the (final) inverse 
motion compensation is applied to tt>e best candidate (identifying connected and . ' 
unconnected pixels). The next part of the image can then be processed, and so on up to a 
complete processing of the whole image. 
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. CLAIMS: 

1. A method of encoding a sequence of frames, composed of picture elements 

(pbceb), by means of a three^lm^sfonal (3D) subband decomposition inwoMng a fiitering 
step applied, in the sequence considered as a 3D volume, to the spatial-temporal data 
which correspond in said sequence to each one of successive groups of ftames (GOft), 
these GOfe being themselves subdivided Into successive pairs of frames (POft) Including a 
so^alled previous frame and a soolled current frame, said decomposition being applied 
to said GOft together with motion estimation and compensation steps performed in each 
GOF on saids POFs and on corresponding pairs of tow-frequency temporal subbands (POSs) 
obtained at each temporal decomposition level, this prxxess of moUbn compensated 
temporal filtering leading in the previous frames on the one hand to connected pixels, that 
are filtered along a motion trajectory corresponding to motion vectors deflned by means of 
said motion estimation steps, and.on the other hand to a residual number of soolled 
unconnected pixels, that are not ffltered at all, each motion estimation step comprising a 
motion search provide for retoming a moUon vector that minimizes a cost function 
depending at least on a distorsion criterion involving a distortion mpasure, said measure 
distorsion bang also applied to the set of said unconnected pb(els. 
2. An encoding method according to daim l,.in which said motion seaixrh is 

provided for minimizing the following ecpression (1) : 

J(m) = SAD(s.cim)HA^.R(m-p) (i) 

with m=(m„m,y being the moHon vector, p^(p,.p,f being the p,«dfctibn for the 

motion vector, being the Lagrange multiplier, the rate term R(m-p) 

representing the motion Infomratton only, and 3MD used as distortion measure being 
computed as : 

SADis.c(m))= j:\4x'y]-clx-m^,y-m,]\ (2) 

with s being the original video signal, c being the coded video signal and B being the 
block Size, and In which the distorsfon criterion extends equation (1), teking into account 
ttie unconnected pixels phenomenon for the minimizing operation that is now applied to 
the following expression (3) : 

K(m) = J(in) + ;it,/ra»wfaH> • ^(■^wvmwfCTOC'n)) « 
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with ^{SyffcomECTSDO^)) bsing the distortion maasune for the set 5uvco«w2C7h> o'' 



uncxjnnected pixels resulting from the motion vector m 
3. An encoding method according to claim 2, said method including, for taking 

into account the distortlbn due to the unconnected pixels, the following steps, successively 
applied to each part of the whole image to be moHon^mpensated 



candidate m, a temporary inverse motion compensation is applied ; 

(b) the set of unconnected pixels is identified ; 

(c) D(SoNcoNNEcrED (m)) is evaluated ; 

(d) the current K(m) value is computed and compared to the current minimum 
value K^{m) to check if the motion vector candidate brings a lower K(m) value ; 

(e) when all the candidates have been tested, a final inveree motion' 
compensation is applied to the best candidate ; 

(f) the steps (a) to (e) are then applied to the next part of the image that can 
be similarly processed, said part of the image being a pixel, a block of pii<els, a maooblbck 
of pDcels or any region provided that the set of parts covers the whole image without any 
overlappihg. 
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Abstract *^ 

^ neans s«d „c«» «,™«,„ steps, »d on «4 ha™. 

»toacoo«tte»co™»cWptelsph™TO^fc,a«,^l^^^ ' 



Fig.l 



